Dynamic Time Warping-based multiple imputation
Or
A series of desparate pleadings with the time series gods

Danielle McCool

12/6/2021

Getcha up to speed

Me (Danielle)

UU and CBS

Last year of my PhD

The pitch

Mobility surveys

What if we used sensors?

A mobile phone app!

Lessons

Participants participate

More (short) trips with the app

Short gaps? Interpolate!

Everybody hates long gaps

Fifteen minutes isn’t long, so I’m going to rush this part a bit. I’m Danielle McCool, and I’m in my last year of my PhD at Utrecht University and Statistics Netherlands.

Circa 2017, a joint effort from both institutions started working on an idea for innovating on mobility surveys – the kind that we send out to people asking them to note all the travel they’ve done on a given day, along with details about it. Shouldn’t we be able to make it better by using the computers we’re all carrying around with us every day? In 2018, an app was released and a field test run, and then I got to play with the data.

Working on my third paper now, if I had to sum up the results of the first two, it’d be something like this: participation for the app wasn’t much worse than your average survey despite providing much more data. We found what we expected regarding our ability to capture more trips than people recall when asked. When it comes to missing data, the only easy answer is not to have any, but short gaps, say under 10 minutes, can be imputed without a whole lot of risk.

Travel behavior as we’d like it

Time series of coordinates
Very precise
Very accurate
Very useful

Travel as we often get it

Incomplete time series of coordinates
Very precise
Unknown accuracy
Usefulness?

Guessing smart on missing data

Take a hint from other disciplines
Multiple imputation

How to understand multiple imputation

The plan

Imputing someone’s exact location

The plan

~~Imputing someone’s exact location~~

The plan

Imputing aggregate characteristics

Distance
Mode of transportation
Radius of gyration

The plan

Imputing aggregate characteristics

Distance
Mode of transportation
Radius of gyration

The plan

Imputing aggregate characteristics

Distance
Mode of transportation
Radius of gyration

The plan pt 2

Imputing aggregate characteristics
Incorporating time as an element

Background photo created by stockking - www.freepik.com

The time series data

Distance in Km per hour by user
	8h	9h	10h	11h	12h	13h	14h	15h	16h	17h	18h	19h
id1	0.0	0.3	0.6	0.1	0.9	0.5	0.0	0.0	3.9	3.0	1.0	0.0
id2	0.0	0.1	0.5	0.0	1.2	1.0	0.5	0.0	4.1	2.3	1.9	0.7
id3	0.0	0.0	0.6	3.7	0.0	2.9	0.0	2.5	0.3	0.0	0.0	0.0
id4	7.0	0.0	0.0	0.0	0.0	0.0	0.0	1.1	0.0	1.3	1.1	4.5
id5	1.6	0.1	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	7.9	0.0
id6	1.5	4.2	0.0	0.4	0.0	0.0	0.1	0.0	2.1	0.0	7.1	4.8

To cumuluative or not to cumulative

Distance of first 5 24-hour days.

Cum. distance in meters of first 5 24-hour days.

Deepdive

Cum. distance in meters of first 5 24-hour days.

We’ve fixed the missing data problem

Cum. distance in meters of first 2 24-hour days.

Oh, right

Cum. distance in meters of two days from one person.

But maybe there’s something to this

Cum. distance in meters of two days from two different people.